Method for encoding and decoding video signal

ABSTRACT

Disclosed is a method for encoding a decoding a video signal. In the procedure of encoding the video signal, when a frame temporarily simultaneous with a frame including a macro block of an enhanced layer which will obtain a prediction video does not exist in a base layer, the macro block is encoded based on difference values of residual data using corresponding residual blocks in a past frame and a future frame of the base layer which are residual data corresponding to image difference values and using a residual block for the macro block of the enhanced layer. In another embodiment, the macro block is encoded based on difference values of residual data using corresponding residual blocks in a past frame and a future frame of the enhanced layer and the residual block for the macro block. Accordingly, a residual prediction mode is applied for a macro block of an enhanced layer even if a frame temporally simultaneous with a frame of the enhanced layer does not exist in a base layer, thereby improve coding efficiency.

This application is a continuation of and claims priority under 35U.S.C. § 120 to co-pending application Ser. No. 11/293,167 “METHOD FORENCODING AND DECODING VIDEO SIGNAL” filed Dec. 5, 2005, the entirety ofwhich is incorporated by reference. This application also claimspriority under 35 U.S.C. §119 on U.S. Provisional Application No.60/632,994, filed on Dec. 6, 2004; the entire contents of which arehereby incorporated by reference.

PRIORITY INFORMATION

This application claims priority under 35 U.S.C. §119 on Korean PatentApplication No. 10-2005-0057565, filed on Jun. 30, 2005, the entirecontents of which are hereby incorporated by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a method for encoding and decoding avideo signal, and more particularly to a method for encoding a videosignal by employing a residual prediction mode and decoding the encodedvideo data.

2. Description of the Prior Art

It is difficult to allocate a broadband available for TV signals towirelessly transmitted/received digital video signals wirelesslytransmitted/received from/in a portable phone and a notebook computer,which have been extensively used, and a mobile TV and a hand held PC,which are expected to be extensively used in the future. Accordingly, astandard to be used for a video compression scheme for such portabledevices must enable a video signal to be compressed with a relativelyhigh efficiency.

In addition, such portable mobile devices are equipped with variousprocessing and presentation capabilities. Accordingly, compressed videosmust be variously prepared corresponding to the capabilities of theportable devices. Therefore, the portable devices must be equipped withvideo data having various qualities obtained through the combination ofvarious parameters including the number of transmission frames persecond, resolution, and the number of bits per pixel with respect to onevideo source, burdening content providers.

For this reason, the content provider prepares compressed video datahaving a high bit rate with respect to one video source so as to providethe portable devices with the video data by decoding the compressedvideo and then encoding the decoded video into video data suitable for avideo processing capability of the portable devices requesting the videodata. However, since the above-described procedure necessarily requirestrans-coding (decoding +scaling +encoding), the procedure causes a timedelay when providing the video requested by the portable devices. Inaddition, the trans-coding requires complex hardware devices andalgorithms due to the variety of a target encoding.

In order to overcome these disadvantages, there is suggested a ScalableVideo Codec (SVC) scheme. According to the SVC scheme, a video signal isencoded with a best video quality in such a manner that the videoquality can be ensured even though parts of the overall picturesequences (frame sequences intermittently selected from among theoverall picture sequences) derived from the encoding are decoded.

A motion compensated temporal filter (or filtering) (MCTF) is anencoding scheme suggested for the SVC scheme. The MCTF scheme requireshigh compression efficiency, that is, high coding efficiency in order tolower the number of transmitted bits per second because the MCTF schemeis mainly employed under a transmission environment such as mobilecommunication having a restricted bandwidth.

As described above, although it is possible to ensure video quality evenif only a part of the sequence of a picture encoded through the MCTF,which is a kind of the SVC scheme, is received and processed, videoquality may be remarkably degraded if a bit rate is lowered. In order toovercome the problem, an additional assistant picture sequence having alow transmission rate, for example, a small-sized video and/or a picturesequence having the smaller number of frames per second may be provided.

The assistant picture sequence is called a base layer, and a mainpicture sequence is called an enhanced (or enhancement) layer. Since thebase layer and the enhanced layer are obtained by encoding the samevideo contents with different spatial resolution and frame rates,redundancy information exists in video signals of both layers.Accordingly, in order to improve coding efficiency of the enhancedlayer, a variety of schemes for predicting the frame of the enhancedlayer based on the frame of the base layer have been suggested.

For example, there is a scheme for coding a motion vector of an enhancedlayer picture by using a motion vector of a base layer picturetemporally simultaneous with the motion vector of the enhanced layerpicture. In addition, it is possible to make a prediction video for avideo frame of the enhanced layer on the basis of the video frame of thebase layer temporally simultaneous with the video frame of the enhancedlayer.

In addition, an additional prediction operation may be performed withrespect to the prediction video of the enhanced layer created inrelation to the main picture sequence by using a prediction video of thebase layer created in relation to the assistance picture sequence. Thisis called a “residual prediction” mode. Herein, the prediction videodenotes an image difference value found by performing a predictionoperation for a macro block. In other words, the prediction videodenotes a video having residual data. Hereinafter, a macro block havinga residual data is called a “residual block”, and a frame having theresidual data is called a “residual frame”.

In more detail, a residual block of an enhanced layer is found through aprediction operation for a macro block in a predetermined frame of amain picture sequence, and the prediction operation is performed evenfor the assistance picture sequence, thereby creating the residual blockand the residual frame of the base layer. Thereafter, a residual blockof the base layer corresponding to the macro block is found. Theresidual block of the base layer undergoes up-sampling, so that the sizeof the residual block of the base layer is enlarged corresponding to thesize of the macro block. The pixel values of the enlarged residual blockof the base layer are subtracted from the pixel values of the residualblock of the enhanced layer, and the resultant value is encoded for themacro block.

FIG. 1 illustrates the conventional residual prediction mode based on amacro block. Herein, an enhanced layer has a frame rate of 30 Hz andframe resolution of CIF. In contrast, a base layer has a frame rate of15 Hz and frame resolution of QCIF.

A fourth residual block R_MB_4 and a fifth residual block R_MB_5 of thebase layer corresponding to a first residual block R_MB_1 and a thirdresidual block R_MB_3 encoded using residual data in the enhanced layerare undergone up-sampling such that the resolution of the residualblocks of the base layer are enhanced to the CIF. Based on the residualblocks of the base layer having the enhanced resolution, the predictionoperation for the residual block of the enhanced layer is performed.

However, since a residual block of the base layer corresponding to thesecond residual block of the enhanced layer R_MB_2 may not exist, it isdifficult to find a predicted video based on the residual block of thebase layer.

In other words, in order to apply the residual prediction mode, aresidual block of the base layer corresponding to a macro block of theenhanced layer must exist. In addition, the residual prediction modecannot be applied even if a residual frame temporally simultaneous witha frame including a macro block of the enhanced layer does not exist inthe base layer.

Accordingly, when the enhanced layer has a frame rate higher than thatof the base layer, a frame temporally simultaneous with a frameincluding a macro block of the enhanced layer to be encoded usingresidual data or difference values of residual data may not exist in thebase layer. Such a frame, which does not exist in the base layer, iscalled a ‘missing picture’. A residual prediction mode cannot be appliedto the macro block of the enhanced layer if the base layer has themissing picture. Accordingly, it is difficult to expect the improvementof coding efficiency.

SUMMARY OF THE INVENTION

Accordingly, the present invention has been made to solve theabove-mentioned problems occurring in the prior art, and an object ofthe present invention is to provide a method for encoding a video signalby applying a residual prediction mode for a macro block of an enhancedlayer even if a frame temporally simultaneous with a frame of theenhanced layer does not exist in a base layer and a method for decodingthe encoded video signal to improve coding efficiency.

To accomplish the above objects, there is provided a method for encodinga video signal, the method comprising the steps of generating a bitstream of a first layer by encoding the video signal through a firstpredetermined scalable scheme, and generating a bit stream of a secondlayer by encoding the video signal through a second predeterminedscheme, wherein the step of generating the bit stream of the first layerincludes a first step of generating a prediction video of a video blockusing frames selected from frames of the second layer, in which theframes consist of blocks encoded in such a manner that the blocks haveresidual data corresponding to difference values of pixels between thereference block and the encoded blocks and include a past frame and afuture frame with respect to a predetermined frame of the first layerhaving the video block encoded using residual blocks having the residualdata.

The first step comprises the sub-steps of generating an insert blocktemporally simultaneous with the predetermined frame by usingcorresponding blocks, which are included in the past frame and thefuture frame of the second layer and have relative positions identicalto a relative position of the video block in a frame, and enlarging asize of the insert block corresponding to a size of the video block ifthere is no frame in the second layer temporally simultaneous with thepredetermined frame, and finding difference values of residual data forthe video block by subtracting pixel values of the enlarged insert blockfrom pixel values of the residual block.

The insert block is generated by weighting a first corresponding blockin the past frame of the second layer and a second corresponding bock inthe future frame of the second layer, respectively.

The method further comprises a step of recording first information on aheader of the video block indicating that the video block is encodedusing difference values of residual data based on the past frame and thefuture frame of the second layer.

According to another aspect of the present invention, there is provideda method for encoding a video signal, the method comprising the steps of(1) generating a residual block having residual data, which aredifference values of pixels between a video block included in apredetermined frame of the video signal and a reference block for thevideo block, and (2) generating a prediction video for the video blockusing a past frame and a future frame of the predetermined frame, theprediction video including blocks encoded in such a manner that theblocks have the residual data.

According to another aspect of the present invention, step (2) includesthe steps of (2-1) generating an insert block temporally simultaneouswith the predetermined frame by using corresponding blocks, which areincluded in the past frame and the future frame on the second layer andhave relative positions identical to a position of the video block in aframe, and (2-2) finding difference values of residual data for thevideo block by subtracting pixel values of the insert block from pixelvalues of the residual block.

According to still another aspect of the present invention, there isprovided a method for decoding an encoded video bit stream, the methodcomprising the steps of decoding a bit stream of a second layer scalablyencoded through a second predetermined scheme, and decoding a bit streamof a first layer encoded through the second predetermined scheme usingdecoding information extracted from the bit stream of the second layer,wherein the step of decoding the bit stream of the first layer includesa first step of restoring a video block, which is included in apredetermined frame of the first layer and encoded using differencevalues of residual data, such that the video block has original pixelvalues by using frames, which are a past frame and a future frame withrespect to the predetermined frame of the first layer, selected fromframes of the second layer and consisting of blocks encoded to haveresidual data, or by using frames, which are a past frame and a futureframe with respect to the predetermined frame of the first layer,selected from frames of the first layer and consisting of blocks encodedto have residual data.

According to yet another aspect of the present invention, there isprovided a method for decoding an encoded video bit stream, the methodcomprising the steps of (1) determining if a video block included in apredetermined frame of the bit stream scalably encoded through apredetermined scheme is encoded using difference values of residualdata, and (2) restoring the video block to a block having original pixelvalues by using a past frame and a future frame of the predeterminedframe, the past frame and the future frame including blocks encoded insuch a manner that the blocks have residual data when the video block isencoded using difference values of residual data.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other objects, features and advantages of the presentinvention will be more apparent from the following detailed descriptiontaken in conjunction with the accompanying drawings, in which:

FIG. 1 is a schematic view illustrating the conventional residualprediction mode based on a macro block;

FIG. 2 is a block diagram illustrating the structure of an image signalencoding device employing a scalable coding scheme for a video signalaccording to an embodiment of the present invention;

FIG. 3 is a view illustrating temporal decomposition with respect to avideo signal in some one temporal decomposition level;

FIG. 4 is a view illustrating a case in which a residual prediction modeis applied to a macro block of an enhanced layer by using a pastresidual frame and a future residual frame in a base layer of a frameincluding the macro block of the enhanced layer according to anembodiment of the present invention;

FIG. 5 is a view illustrating a case in which a residual prediction modeis applied to a macro block of an enhanced layer by using a pastresidual frame and a future frame in an enhanced layer of a frameincluding the macro block of the enhanced layer according to anotherembodiment of the present invention; and

FIG. 6 is a block diagram illustrating the structure of a decoder fordecoding a data stream encoded by the device shown in FIG. 2.

FIG. 7 is a block diagram illustrating the structure of a decoder forexecuting an MCTF scheme according to one exemplary embodiment of thepresent invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Hereinafter, preferred embodiments of the present invention will bedescribed with reference to the accompanying drawings. In the followingdescription and drawings, the same reference numerals are used todesignate the same or similar components, and so repetition of thedescription on the same or similar components will be omitted.

FIG. 2 is a block diagram illustrating the structure of a video signalencoding device employing a scalable coding scheme for a video signalaccording to the present invention.

The video signal encoding device shown in FIG. 2 includes an enhancedlayer (EL) encoder 100 for scalably encoding an input video signal basedon a macro block through a Motion Compensated Temporal Filter (MCTF)scheme and generating suitable management information, a texture codingunit 110 for converting the encoded data of each macro block into acompressed bit string, a motion coding unit 120 for coding motionvectors of a video block obtained from the EL encoder 100 into acompressed bit string through a specific scheme, a base layer encoder150 for encoding an input video signal through a predetermined schemesuch as the MPEG 1, 2, 4, H.261, or H.264 and generating the sequence ofsmall-sized videos, for example, the sequence of pictures having 25% ofan original video size if necessity, a muxer 130 for encapsulating theoutput data of the texture coding unit 110, the picture sequence of theBL encoder 150, and an output vector data of the motion coding unit 120in a predetermined format, multiplexing the data with each other in apredetermined format, and then outputting the multiplexed data.

The EL encoder 100 performs a prediction operation for subtracting areference block obtained through motion estimation from a macro block ina predetermined video frame (or picture) and performs an updateoperation by adding the image difference between the macro block and thereference block to the reference block. In addition, the EL encoder 100may additionally perform a residual prediction operation with respect tothe macro block representing the image difference with regard to thereference block by using base layer data.

The EL encoder 100 divides the sequence of input video frames intoframes, which will have image difference values, and frames, to whichthe image difference values will be added. For example, the EL encoder100 divides the input video frames into odd frames and even frames.Then, the EL encoder 100 performs the prediction operation, the updateoperation, and the residual predication operation with respect to, forexample, one group of pictures (GOP) through several levels until thenumber of L frames (frames generated through the update operation)becomes one. FIG. 3 illustrates the structure relating to the predictionoperation, the update operation, and the residual predication operationin one of the above levels.

The structure shown in FIG. 3 includes a BL decoder 105, for extractingencoded information including a frame rate of a base layer stream forthe small-sized image sequence encoded in the BL encoder 150 anddecoding the base layer stream, an estimation/prediction unit 101 forestimating a reference block for each macro block included in a frame,which may have residual data through motion estimation, that is an oddframe, in even frames provided before or after the odd frame(inter-frame mode) or in its own frame (intra mode) and performing aprediction motion for calculating a motion vector and/or a imagedifference between the macro block and the reference block (differencevalues between corresponding pixels), an update unit 102 for performingthe update operation through which an image difference calculated withrespect to the macro block is normalized and the normalized imagedifference is added to a corresponding reference block in the adjacentframe (e.g., the even frame) including the reference block for the macroblock, and a residual prediction unit 103 for performing an additionalprediction operation, that is, a residual prediction operation by usinga residual block in the enhanced layer or the base layer correspondingto the residual block having residual data (image difference) generatedthrough the prediction operation of the estimation/prediction unit 101.

The operation performed by the estimation/prediction unit 101 is calleda “P” operation, a frame generated through the P operation is called an“H” frame, and residual data existing in the H frame reflects a harmoniccomponent of a video signal. In addition, the operation performed by theupdate unit 102 is called a “U” operation, a frame generated through theU operation is called an “L” frame, and the L frame has a low sub-bandpicture.

The estimation/prediction unit 101, the update unit 102, and theresidual predication unit 103 shown in FIG. 3 can parallely andsimultaneously process a plurality of slices divided from one frameinstead of a frame unit. In the following description, the term “frame”can be replaced with the “slices” if it does not make technicaldifference, that is, the frame includes the meaning of the slices.

The estimation/prediction unit 101 divides input video frames or oddframes of L frames obtained through all levels into macro blocks havinga predetermined size, searches temporally adjacent even frames in thesame temporal decomposition level for blocks having the most similarimages to images of divided macro blocks, makes a prediction video ofeach macro block based on the searched block, and finds a motion vectorof the macro block. If a block having correlation above a suitablethreshold value is not searched, that is, if a reference block is notsearched, encoding is performed in the internal mode with respect to acurrent macro block using adjacent pixel values.

A block having the highest correlation has the smallest image differencebetween the block and a target block. The image difference is determinedas the sum of pixel-to-pixel difference values or the average of thesum. The smallest macro block (the smallest macro blocks among blocks)having at most a predetermined threshold value is (are) called areference block (reference blocks).

If the reference block is found, the estimation/prediction unit 101finds a motion vector to the reference block from the current macroblock to be delivered to the motion coding unit 120 and calculates apixel difference value between each pixel value of the reference block(in a case of one frame) and each pixel value of the current macroblock, or a pixel difference value between each pixel average value ofthe reference block (in a case of plural frames) and the pixel value ofthe current macro block, thereby encoding a corresponding macro block.In addition, the estimation/prediction unit 101 inserts a relativedistance between a frame including the selected reference block and aframe including the current macro block and/or one of reference blockmodes such as a Skip mode, a DirInv mode, a Bid mode, a Fwd mode, a Bwdmode, and an intra mode into a header field of the corresponding macroblock.

The estimation/prediction unit 101 performs the procedure with respectto all macro blocks in a frame, thereby making an H frame which is aprediction video for the frame. In addition, the estimation/predictionunit 101 makes H frames, which are prediction videos for frames, withrespect to input video frames or all odd frames of L frames obtainedthrough all levels.

As described above, the update unit 102 adds image difference values formacro blocks in the H frame generated by the estimation/prediction unit101 to L frames (input video frames or even frames of L frames obtainedthrough all levels) having corresponding reference blocks.

In the meantime, the residual prediction unit 103 searches for aresidual block of the base layer corresponding to a residual block (foran enhanced layer) of an H frame (residual frame) having residual datagenerated in the estimation/prediction unit 101 and enlarging theresidual block of the base layer through up-sampling, and subtracts thepixel values of the enlarged residual block of the base layer from thepixel values of the residual block of the enhanced layer, therebyadditionally enabling a residual prediction operation.

However, if a residual frame temporally simultaneous with a currentframe including a residual block of the enhanced layer does not exist inthe base layer output in the BL decoder 105, there is no residual blockof the base layer corresponding to the residual block of the enhancedlayer.

In this case, the residual prediction unit 103 according to the presentinvention may perform a residual prediction operation using the residualframe of the enhanced layer or an adjacent residual frame of the baselayer.

For example, the residual prediction unit 103 may perform the residualprediction operation based on a past frame and a future frame of acurrent frame including the residual block of the enhanced layer fromamong residual frames of the base layer.

In addition, in the case in which there is no the residual block of thebas layer corresponding to the residual block of the enhanced layer, theresidual prediction unit 103 according to the another embodiment of thepresent invention may perform the residual prediction operation based onthe past frame and the future frame of the current frame including theresidual block from among residual frames of the enhanced layer, thatis, H frames generated by the estimation/prediction 101.

Hereinafter, embodiments according to the present invention employing aresidual estimation mode for the residual block of the enhanced layerwhen a residual frame temporally simultaneous with the current frameincluding the residual block of the enhanced layer does not exist in thebase layer will be described with reference to FIGS. 4 and 5.

FIG. 4 is a view illustrating a case in which a residual estimation modeis applied to a macro block of an enhanced layer by using a pastresidual frame and a future residual frame in a base layer of a frameincluding the macro block of the enhanced layer according to anembodiment of the present invention.

The embodiment shown in FIG. 4 may be employed for a case in which aresidual frame temporally simultaneous with a current frame including aresidual block R_MB_2 of an enhanced layer does not exist in a baselayer. The existence of the simultaneous residual frame in the baselayer is determined by determining if a frame having the same pictureorder count (POC), which is the current frame, exists in the base layerbased on the POC informing a picture decoding order from among encodinginformation extracted from the BL decoder 105.

The residual prediction unit 103 creates an insert residual block of thebase layer temporally simultaneous with a residual block of the enhancedlayer by weighting pixel values of residual blocks R_MB_4 and R_MB_5(the residual blocks have the same relative positions in frames), whichcorrespond to the residual block R_MB_2 of the enhanced layer and areincluded in the past residual frame and the future residual frame of thebase layer, using a suitable weight value. Then, the residual predictionunit 103 enlarges the size of the insert residual block of the baselayer to the size of the macro block of the enhanced layer byup-sampling the insert residual block.

Herein, the weight values are values between 0 and 1 selected in such amanner that the sum of the two weight values is equal to 1. The weightvalues may be determined as values proportional to a correlation betweena frame including the residual block of the enhanced layer and the pastresidual frame or the future residual frame. In addition, the weightvalues may be determined as values inversely proportional to a temporaldistance between the frame including the residual block of the enhancedlayer and the past residual frame or the future residual frame.

The residual prediction unit 103 subtracts pixel values of the enlargedinsert residual block of the base layer from pixel values of theresidual block R_MB_2 of the enhanced layer, thereby making new residualdata for a corresponding macro block of the enhanced layer, whichcorresponds to a different value between the residual data.

The residual prediction unit 103 may insert a “residual_prediction_flag”set to, for example, 1 into a header field of the current macro block,so that the decoder can be notified that the residual prediction mode isapplied to the current macro block, that is, the current macro block isencoded by using the difference value between the residual data.

In addition, the residual prediction unit 103 inserts a“residual_direction_flag” set to, for example, 1 into the header fieldof the current macro block, so that it can be indicated that thedifference value of residual data for the current macro block iscalculated by using the insert residual block of the base layer createdbased on the corresponding blocks of the past residual frame and thefuture residual frame of the base layer. The residual prediction unit103 can insert one of weight values used for creating the insertresidual block (e.g., a weight value used when a corresponding residualblock of the past residual frame is weighted) into the header field ofthe current macro block.

FIG. 5 is a view illustrating a case in which a residual estimation modeis applied to a macro block of an enhanced layer by using a pastresidual frame and a future frame (‘H’ frames) in an enhanced layer withrespect to a frame including the macro block of the enhanced layeraccording to another embodiment of the present invention. The embodimentshown in FIG. 5 can be applied to a case in which a residual frametemporally simultaneous with the frame (a current H frame) including theresidual block of the enhanced layer does not exist in the base layer.

The residual prediction unit 103 creates an insert residual block of theenhanced layer temporally simultaneous with the current H frame byweighting suitable weight values to pixel values of the correspondingresidual blocks R_MB_1 and R_MB_3 corresponding to the residual blockR_MB_2 and included in a past H frame and a future H frame of thecurrent H frame.

Herein, the weight values are values between 0 and 1 selected in such amanner that the sum of the two weight values is equal to 1. The weightvalues may be determined as values proportional to correlation betweenthe current H frame and the past H frame or the future H frame. Inaddition, the weight values may be determined as values inverselyproportional to a temporal distance between the current H frame and thepast H frame or the future H frame.

The residual prediction unit 103 subtracts pixel values of the insertresidual block of the enhanced layer from pixel values of the residualblock R_MB_2, thereby making new residual data for a corresponding macroblock, which is a different value of the residual data.

The residual prediction unit 103 may insert a “residual_prediction_flag”set to, for example, 1 into a header field of the current macro block,so that the decoder can be notified that the residual prediction mode isapplied to the current macro block, that is, the current macro block isencoded using the difference value of the residual data.

In addition, the residual prediction unit 103 may insert a“residual_direction_flag” set to, for example, ‘0’ into the header fieldof the current macro block, so that it can be indicated that thedifference value for the current macro block of residual data iscalculated by using the insert residual block of the enhanced layercreated based on the corresponding blocks of the past frame and thefuture frame of the enhanced layer. The residual prediction unit 103 caninsert one of weight values used for creating the insert residual blockof the enhanced layer (e.g., a weight value used when a correspondingresidual block of the past residual frame is weighted) into the headerfield of the current macro block.

Herein, the residual_prediction_flag of ‘1’ indicates that acorresponding macro block is encoded by using difference values betweenpixel values of a residual block of the enhanced layer for thecorresponding macro block and pixel values of i) a correspondingresidual block of the base layer temporally simultaneous with the macroblock, ii) the insert residual block of the base layer created based onresidual blocks of the past frame and the future frame of the base layeras shown in FIG. 4, or iii) the insert residual block of the enhancedlayer created based on residual blocks of the past frame and the futureframe of the enhanced layer as shown in FIG. 5. In other words,residual_prediction_flag of ‘1’ indicates that the corresponding macroblock is encoded by using difference values of residual data.

In contrast, the residual_prediction_flag of ‘0’ indicates that acorresponding macro block is encoded using difference values betweenpixel values of the corresponding macro block and pixel values of areference block for the macro block, that is, by using residual data. Inother words, this means that only a prediction operation of theestimation/prediction unit 101 is performed, and a prediction operationof the residual prediction unit 103 is not performed with respect to themacro block.

A frame formed by including a macro block encoded based on the newresidual data, that is, difference values of residual data is referredto as a ‘H^(rd)’ frame in order to be distinguished from the H frameformed only by using a macro block encoded based on residual data.

A data stream encoded through the above-described scheme may performwire or wireless transmission to the decoder or transmission to thedecoder by means of storage medium. The decoder recovers an originalvideo signal according to a scheme to be described below.

FIG. 6 is a block diagram illustrating the structure of the decoder fordecoding the data stream encoded by the device shown in FIG. 2. Thedecoder shown in FIG. 6 includes a de-muxer 200 for dividing thereceived data stream into a compressed motion vector stream and acompressed macro block information stream, a texture decoding unit 210for recovering an original uncompressed information stream from thecompressed macro block information stream, a motion decoding unit 220for recovering an original uncompressed stream from a compressed motionvector stream, an enhanced layer (EL) decoder 230 for converting theuncompressed macro block information stream and the motion vector streaminto an original video signal through an MCTF scheme, and a base layer(BL) decoder 240 for decoding base layer stream through a predeterminedscheme such as the MPEG 4 scheme or the H.264 scheme. The EL decoder 230uses base layer encoding information such as the POC and base layer data(a macro block or frame data) directly extracted from the base layerstream, or obtained by inquiring the information and the data from theBL decoder 240.

The EL decoder 230 decodes an input stream into data having an originalframe sequence, and FIG. 7 is a block diagram illustrating the mainstructure of the EL decoder 230 employing the MCTF scheme in detail.

FIG. 7 illustrates the structure performing temporal composition withrespect to the sequence of the H^(rd) frames (or the H frames) in atemporal decomposition level of N and the sequence of L frames so as tomake the sequence of an L frames in a temporal decomposition level ofN−1. The structure shown in FIG. 7 includes a residual inverseprediction unit 235 for converting input ‘H^(rd)’ frames into H frameshaving residual data (image differences) by adding pixel values of macroblocks, encoded through a residual prediction operation using differencevalues of residual data, included in the input ‘H^(rd)’ frames to pixelvalues of residual blocks corresponding to the macro blocks, an inverseupdate unit 231 for selectively subtracting difference values for pixelsof the H frames output from the residual inverse prediction unit 235from pixel values of input L frames, an inverse prediction unit 232 forrecovering L frames having original images using the H frames and Lframes obtained by subtracting the image difference values of the Hframes from the input L frames, a motion vector decoder 233 forproviding motion vector information of each block in the H frames toboth the inverse update unit 231 and the inverse prediction unit 232 ineach stage or each temporal decomposition level, and an arranger 234 formaking a normal L frame sequence by inserting the L frames formed by theinverse prediction unit 232 into the L frames output from the inverseupdate unit 231.

The L frame sequence output by the arranger 234 becomes the sequence ofL frames 701 in a level of N−1 and is restored to the sequence of Lframes by an inverse update unit and an inverse prediction unit in anext stage together with the sequence of input H^(rd) frames or input Hframes in the level of N−1. This procedure is performed by the number oflevels in the encoding procedure, so that the sequence of original videoframes is obtained.

Hereinafter, a recovering procedure (a temporal composition procedure)in the level of N of recovering an L frame in the level of N−1 from thereceived H^(rd) (or the received H frame) in the level of N and the Lframe in the level of N having been generated from the level of N+1 willbe described in more detail.

If the residual_prediction_flag indicating that a macro block in apredetermined H^(rd) frame is encoded using difference values ofresidual data is included in the header of the macro block, the residualinverse prediction unit 235 generates residual data for the macro blockby using the base layer data and the base layer encoding informationextracted from the BL decoder 240 or by using a past frame and a futureframe of the H^(rd) frame.

If the macro block in the predetermined H^(rd) frame is encoded usingdifference values of residual data, the residual inverse prediction unit235 determines based on a POC of the encoding information extracted bythe BL decoder 240 if a frame having the same POC exists in the baselayer, thereby determining if a residual frame temporally simultaneouswith the H^(rd) frame including the macro block exists in the baselayer.

If the simultaneous residual frame exists in the base layer, theresidual inverse prediction unit 235 searches for a residual blockcorresponding to the macro block (the residual block having a relativeposition identical to that of the macro block in the frame) included inthe simultaneous residual frame in the base layer, enlarges the size ofthe residual block to the size of the macro block by up-sampling theresidual block according to necessity, and then adds pixel values of theenlarged corresponding residual block to pixel values of the macro blockencoded using the difference values of residual data, thereby creating amacro block having residual data.

In contrast, if the simultaneous residual frame does not exist in thebase layer, the residual inverse prediction unit 235 determines based onthe residual_direction_flag included in the header of the macro block ifthe encoded difference values of the residual data in the macro blockare calculated based on the insert residual block of the base layer orbased on the insert residual block of the enhanced layer.

If the residual_direction_flag is set to, for example, 1, that is, ifthe encoded difference values of the residual data in the macro blockare calculated based on the insert residual block of the base layer, theresidual inverse prediction unit 235 searches for the correspondingresidual block (the residual block has a relative position identical tothat of the macro block in the frame), which corresponds to the macroblock of the enhanced layer and is included in the past residual frameand the future residual frame of the predetermined H^(rd) frame, fromamong the base layer data provided from the BL decoder 240. Thereafter,the residual inverse prediction unit 235 creates an insert residualblock of the base layer temporally simultaneous with the macro block byweighting the pixel values of the corresponding residual block with afirst weight value included in the header for the macro block and asecond weight value calculated from the first weight value and thenenlarges the size of the insert residual block into the size of themacro block by up-sampling the insert residual block. Thereafter, theresidual inverse prediction unit 235 generates residual data for themacro block by adding pixel values of the enlarged insert residual blockof the base layer to pixel values of the macro block.

In contrast, if the residual_direction_flag is set to 0, that is, if theencoded difference values of the residual data in the macro block arecalculated based on the insert residual block of the enhanced layer, theresidual inverse prediction unit 235 searches for a correspondingresidual block, which has a relative position identical to that of themacro block in the frame and is included in the past residual frame andthe future residual frame of the predetermined H^(rd) frame, creates aninsert residual block of the enhanced layer temporally simultaneous withthe macro block by weighting the pixel values of the correspondingresidual block having residual data based on a first weight valueincluded in the header for the macro block and a second weight valuecalculated from the first weight value, and then generates residual datafor the macro block by adding pixel values of the insert residual blockof the enhanced layer to pixel values of the macro block.

Through the above-described procedure, the residual inverse predictionunit 235 performs a residual inverse prediction operation with respectto the macro block encoded using the difference values of residual dataso as to be included in an H^(rd) frame, thereby creating an H frameformed only by using residual data, that is, macro blocks havingdifference values.

In the meantime, with respect to a predetermined L frame (in the levelof N), in consideration of a motion vector provided from the motionvector decoder 233, the inverse update unit 231 detects an H frame (inthe level of N) having image difference found using a block in anoriginal L frame (in the level of N−1) updated to a predetermined Lframe (in the level of N) through the encoding procedure as a referenceblock and then subtracts image difference values for the macro block inthe H frame from pixel values of the corresponding block in the L frame,thereby recovering an original L frame.

The inverse update operation is performed with respect to a blockupdated using image difference values of a macro block in the H framethrough the encoding procedure from among blocks in the current L frame(in the level of N), so that the L frame in the level of L−1 isrecovered.

In a macro block in a predetermined H frame, the inverse prediction unit232 detects a reference block in an L frame (the L frame isinverse-updated and output by the inverse update unit 231) based on themotion vector provided from the motion vector decoder 233 and then addspixel values of the reference block to difference values of pixels ofthe macro block, thereby recovering original video data.

If original video data are recovered from all macro blocks in thecurrent H frame through the above described operation, and the macroblocks undergo a composition procedure so that an L frame is recovered,the L frame is alternatively arranged together with an L frame, which isrecovered in the inverse update unit 231, through the arranger 234, sothat the arranged frame is output to the next stage.

As described above, a perfect video frame sequence is recovered from theencoded data stream. In particular, when one GOP undergoes N predictionoperations and N update operations through the encoding procedure inwhich the MCTF scheme may be employed, if N inverse update operationsand N inverse prediction operations are performed in an MCTF decodingprocedure, video quality of an original video signal can be obtained. Ifthe operations are performed by the frequency number smaller than N, avideo frame may have relatively smaller bit rates even though the videoquality of the video frame is degraded somewhat as compared with a videoframe through N operations. Accordingly, the decoder is designed toperform the inverse update operation and the inverse predictionoperation suitably for the performance of the decoder.

The above-described decoder may be installed in a mobile communicationterminal or a device for reproducing record media.

As described above, according to the present invention, when a videosignal is scalably encoded, a residual prediction mode is applied for amacro block of an enhanced layer even if a frame temporally simultaneouswith a frame of the enhanced layer does not exist in a base layer,thereby improve coding efficiency.

Although preferred embodiments of the present invention have beendescribed for illustrative purposes, those skilled in the art willappreciate that various modifications, additions and substitutions arepossible, without departing from the scope and spirit of the inventionas disclosed in the accompanying claims.

1. A method for decoding a video signal in a video decoder, the methodcomprising: receiving, with a demuxer, the video signal scalably encodedcorresponding to a base layer and an enhanced layer; obtaining, from thebase layer, with the decoder, first weight information of an image blockand a reference index indicating an enhanced reference block, the imageblock and the enhanced reference block being included in the enhancedlayer, the enhanced reference block being referenced by the image block,and the first weight information being the same as second weightinformation of a corresponding block included in the base layer, thecorresponding block including residual data when the image black isresidual-coded; and decoding, with the decoder, the image block usingthe first weight information and the reference index.
 2. The method ofclaim 1, wherein a spatial resolution of the enhanced layer is higherthan a spatial resolution of the base layer, and wherein the enhancedlayer and the base layer are from a same video signal.
 3. An apparatusfor decoding a video signal, comprising: a demuxer configured to receivethe video signal scalably encoded corresponding to a base layer and anenhanced layer; and an enhanced layer decoder configured to obtain, fromthe base layer, first weight information of an image block and areference index indicating an enhanced reference block and configured todecode the image block using the first weight information and thereference index, the image block and the enhanced reference block beingincluded in the enhanced layer, the enhanced reference block beingreferenced by the image block, and the first weight information beingthe same as second weight information of a corresponding block includedin the base layer, the corresponding reference block including residualdata when the image block is residual-coded.
 4. The apparatus of claim3, wherein a spatial resolution of the enhanced layer is higher than aspatial resolution of the base layer, and wherein the enhanced layer andthe base layer are from a same video signal.